Pandas Profiling Report

Dataset statistics

Number of variables	10
Number of observations	20640
Missing cells	207
Missing cells (%)	0.1%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	1.6 MiB
Average record size in memory	80.0 B

Variable types

Numeric	8
Categorical	2

Alerts

`total_rooms` has a high cardinality: 5832 distinct values	High cardinality
`longitude` is highly correlated with `median_house_value` and 1 other fields	High correlation
`latitude` is highly correlated with `longitude`	High correlation
`total_bedrooms` is highly correlated with `population` and 1 other fields	High correlation
`population` is highly correlated with `total_bedrooms` and 1 other fields	High correlation
`households` is highly correlated with `total_bedrooms` and 1 other fields	High correlation
`median_income` is highly correlated with `median_house_value`	High correlation
`median_house_value` is highly correlated with `longitude` and 2 other fields	High correlation
`ocean_proximity` is highly correlated with `longitude` and 1 other fields	High correlation
`total_bedrooms` has 207 (1.0%) missing values	Missing
`latitude` is highly skewed (γ1 = 59.45812641)	Skewed

Reproduction

Analysis started	2022-10-20 16:40:41.388164
Analysis finished	2022-10-20 16:40:58.528578
Duration	17.14 seconds
Software version	pandas-profiling v3.3.0
Download configuration	config.json

longitude
Real number (ℝ)

HIGH CORRELATION

Distinct	844
Distinct (%)	4.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	-119.5697045

Minimum	-124.35
Maximum	-114.31
Zeros	0
Zeros (%)	0.0%
Negative	20640
Negative (%)	100.0%
Memory size	161.4 KiB

Quantile statistics

Minimum	-124.35
5-th percentile	-122.47
Q1	-121.8
median	-118.49
Q3	-118.01
95-th percentile	-117.08
Maximum	-114.31
Range	10.04
Interquartile range (IQR)	3.79

Descriptive statistics

Standard deviation	2.003531724
Coefficient of variation (CV)	-0.01675618195
Kurtosis	-1.330152366
Mean	-119.5697045
Median Absolute Deviation (MAD)	1.28
Skewness	-0.297801208
Sum	-2467918.7
Variance	4.014139367
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-118.31	162	0.8%
-118.3	160	0.8%
-118.29	148	0.7%
-118.27	144	0.7%
-118.32	142	0.7%
-118.28	141	0.7%
-118.35	140	0.7%
-118.36	138	0.7%
-118.19	135	0.7%
-118.37	128	0.6%
Other values (834)	19202	93.0%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-124.35	1	< 0.1%
-124.3	2	< 0.1%
-124.27	1	< 0.1%
-124.26	1	< 0.1%
-124.25	1	< 0.1%
-124.23	3	< 0.1%
-124.22	1	< 0.1%
-124.21	3	< 0.1%
-124.19	4	< 0.1%
-124.18	6	< 0.1%

Value	Count	Frequency (%)
-114.31	1	< 0.1%
-114.47	1	< 0.1%
-114.49	1	< 0.1%
-114.55	1	< 0.1%
-114.56	1	< 0.1%
-114.57	3	< 0.1%
-114.58	2	< 0.1%
-114.59	2	< 0.1%
-114.6	3	< 0.1%
-114.61	3	< 0.1%

latitude
Real number (ℝ_≥0)

HIGH CORRELATION
SKEWED

Distinct	863
Distinct (%)	4.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	35.64834399

Minimum	32.54
Maximum	378
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	161.4 KiB

Quantile statistics

Minimum	32.54
5-th percentile	32.82
Q1	33.93
median	34.26
Q3	37.71
95-th percentile	38.96
Maximum	378
Range	345.46
Interquartile range (IQR)	3.78

Descriptive statistics

Standard deviation	3.20017739
Coefficient of variation (CV)	0.08977071671
Kurtosis	6345.285734
Mean	35.64834399
Median Absolute Deviation (MAD)	1.23
Skewness	59.45812641
Sum	735781.82
Variance	10.24113533
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
34.06	244	1.2%
34.05	236	1.1%
34.08	234	1.1%
34.07	231	1.1%
34.04	221	1.1%
34.09	212	1.0%
34.02	208	1.0%
34.1	203	1.0%
34.03	193	0.9%
33.93	181	0.9%
Other values (853)	18477	89.5%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
32.54	1	< 0.1%
32.55	3	< 0.1%
32.56	10	< 0.1%
32.57	18	0.1%
32.58	26	0.1%
32.59	11	0.1%
32.6	9	< 0.1%
32.61	14	0.1%
32.62	13	0.1%
32.63	18	0.1%

Value	Count	Frequency (%)
378	1	< 0.1%
41.95	2	< 0.1%
41.92	1	< 0.1%
41.88	1	< 0.1%
41.86	3	< 0.1%
41.84	1	< 0.1%
41.82	1	< 0.1%
41.81	2	< 0.1%
41.8	3	< 0.1%
41.79	1	< 0.1%

housing_median_age
Real number (ℝ_≥0)

Distinct	53
Distinct (%)	0.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	28.65692829

Minimum	1
Maximum	400
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	161.4 KiB

Quantile statistics

Minimum	1
5-th percentile	8
Q1	18
median	29
Q3	37
95-th percentile	52
Maximum	400
Range	399
Interquartile range (IQR)	19

Descriptive statistics

Standard deviation	12.84802111
Coefficient of variation (CV)	0.4483390884
Kurtosis	32.84630803
Mean	28.65692829
Median Absolute Deviation (MAD)	10
Skewness	1.222725723
Sum	591479
Variance	165.0716464
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
52	1273	6.2%
36	862	4.2%
35	824	4.0%
16	771	3.7%
17	698	3.4%
34	689	3.3%
26	619	3.0%
33	615	3.0%
18	570	2.8%
25	566	2.7%
Other values (43)	13153	63.7%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
1	4	< 0.1%
2	58	0.3%
3	62	0.3%
4	191	0.9%
5	244	1.2%
6	160	0.8%
7	175	0.8%
8	206	1.0%
9	205	1.0%
10	264	1.3%

Value	Count	Frequency (%)
400	1	< 0.1%
52	1273	6.2%
51	48	0.2%
50	136	0.7%
49	134	0.6%
48	177	0.9%
47	198	1.0%
46	245	1.2%
45	294	1.4%
44	356	1.7%

total_rooms
Categorical

HIGH CARDINALITY

Distinct	5832
Distinct (%)	28.3%
Missing	0
Missing (%)	0.0%
Memory size	161.4 KiB

??	1018
1527	18
1613	17
1471	15
2127	15
Other values (5827)	19557

Length

Max length	5
Median length	4
Mean length	3.802713178
Min length	1

Characters and Unicode

Total characters	78488
Distinct characters	11
Distinct categories	2 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	2018 ?
Unique (%)	9.8%

Sample

1st row	880
2nd row	7099
3rd row	1467
4th row	1274
5th row	1627

Common Values

Value	Count	Frequency (%)
??	1018	4.9%
1527	18	0.1%
1613	17	0.1%
1471	15	0.1%
2127	15	0.1%
2053	15	0.1%
1717	15	0.1%
1705	14	0.1%
1787	14	0.1%
1607	14	0.1%
Other values (5822)	19485	94.4%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
	1018	4.9%
1527	18	0.1%
1613	17	0.1%
1471	15	0.1%
2127	15	0.1%
2053	15	0.1%
1717	15	0.1%
1650	14	0.1%
1731	14	0.1%
1703	14	0.1%
Other values (5822)	19485	94.4%

Most occurring characters

Value	Count	Frequency (%)
1	13023	16.6%
2	11259	14.3%
3	8334	10.6%
4	7184	9.2%
5	6548	8.3%
7	6215	7.9%
6	6129	7.8%
0	5946	7.6%
9	5908	7.5%
8	5906	7.5%

Most occurring categories

Value	Count	Frequency (%)
Decimal Number	76452	97.4%
Other Punctuation	2036	2.6%

Most frequent character per category

Decimal Number

Value	Count	Frequency (%)
1	13023	17.0%
2	11259	14.7%
3	8334	10.9%
4	7184	9.4%
5	6548	8.6%
7	6215	8.1%
6	6129	8.0%
0	5946	7.8%
9	5908	7.7%
8	5906	7.7%

Other Punctuation

Value	Count	Frequency (%)
?	2036	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Common	78488	100.0%

Most frequent character per script

Common

Value	Count	Frequency (%)
1	13023	16.6%
2	11259	14.3%
3	8334	10.6%
4	7184	9.2%
5	6548	8.3%
7	6215	7.9%
6	6129	7.8%
0	5946	7.6%
9	5908	7.5%
8	5906	7.5%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	78488	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
1	13023	16.6%
2	11259	14.3%
3	8334	10.6%
4	7184	9.2%
5	6548	8.3%
7	6215	7.9%
6	6129	7.8%
0	5946	7.6%
9	5908	7.5%
8	5906	7.5%

total_bedrooms
Real number (ℝ_≥0)

HIGH CORRELATION
MISSING

Distinct	1923
Distinct (%)	9.4%
Missing	207
Missing (%)	1.0%
Infinite	0
Infinite (%)	0.0%
Mean	537.8705525

Minimum	1
Maximum	6445
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	161.4 KiB

Quantile statistics

Minimum	1
5-th percentile	137
Q1	296
median	435
Q3	647
95-th percentile	1275.4
Maximum	6445
Range	6444
Interquartile range (IQR)	351

Descriptive statistics

Standard deviation	421.3850701
Coefficient of variation (CV)	0.7834321252
Kurtosis	21.98557506
Mean	537.8705525
Median Absolute Deviation (MAD)	162
Skewness	3.459546332
Sum	10990309
Variance	177565.3773
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
280	55	0.3%
331	51	0.2%
345	50	0.2%
343	49	0.2%
393	49	0.2%
348	48	0.2%
394	48	0.2%
328	48	0.2%
309	47	0.2%
272	47	0.2%
Other values (1913)	19941	96.6%
(Missing)	207	1.0%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
1	1	< 0.1%
2	2	< 0.1%
3	5	< 0.1%
4	7	< 0.1%
5	6	< 0.1%
6	5	< 0.1%
7	6	< 0.1%
8	8	< 0.1%
9	7	< 0.1%
10	8	< 0.1%

Value	Count	Frequency (%)
6445	1	< 0.1%
6210	1	< 0.1%
5471	1	< 0.1%
5419	1	< 0.1%
5290	1	< 0.1%
5033	1	< 0.1%
5027	1	< 0.1%
4957	1	< 0.1%
4952	1	< 0.1%
4819	1	< 0.1%

population
Real number (ℝ)

HIGH CORRELATION

Distinct	3889
Distinct (%)	18.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	1425.379942

Minimum	-999
Maximum	35682
Zeros	0
Zeros (%)	0.0%
Negative	1
Negative (%)	< 0.1%
Memory size	161.4 KiB

Quantile statistics

Minimum	-999
5-th percentile	348
Q1	787
median	1166
Q3	1725
95-th percentile	3288
Maximum	35682
Range	36681
Interquartile range (IQR)	938

Descriptive statistics

Standard deviation	1132.583966
Coefficient of variation (CV)	0.794583909
Kurtosis	73.52288283
Mean	1425.379942
Median Absolute Deviation (MAD)	440
Skewness	4.934049079
Sum	29419842
Variance	1282746.44
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
891	25	0.1%
1227	24	0.1%
1052	24	0.1%
850	24	0.1%
761	24	0.1%
825	23	0.1%
782	22	0.1%
1005	22	0.1%
872	21	0.1%
999	21	0.1%
Other values (3879)	20410	98.9%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-999	1	< 0.1%
3	1	< 0.1%
5	1	< 0.1%
6	1	< 0.1%
8	4	< 0.1%
9	2	< 0.1%
11	1	< 0.1%
13	4	< 0.1%
14	3	< 0.1%
15	2	< 0.1%

Value	Count	Frequency (%)
35682	1	< 0.1%
28566	1	< 0.1%
16305	1	< 0.1%
16122	1	< 0.1%
15507	1	< 0.1%
15037	1	< 0.1%
13251	1	< 0.1%
12873	1	< 0.1%
12427	1	< 0.1%
12203	1	< 0.1%

households
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	1815
Distinct (%)	8.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	499.5396802

Minimum	1
Maximum	6082
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	161.4 KiB

Quantile statistics

Minimum	1
5-th percentile	125
Q1	280
median	409
Q3	605
95-th percentile	1162
Maximum	6082
Range	6081
Interquartile range (IQR)	325

Descriptive statistics

Standard deviation	382.3297528
Coefficient of variation (CV)	0.7653641301
Kurtosis	22.05798806
Mean	499.5396802
Median Absolute Deviation (MAD)	151
Skewness	3.410437712
Sum	10310499
Variance	146176.0399
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
306	57	0.3%
386	56	0.3%
335	56	0.3%
282	55	0.3%
429	54	0.3%
375	53	0.3%
284	51	0.2%
297	51	0.2%
278	50	0.2%
340	50	0.2%
Other values (1805)	20107	97.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
1	1	< 0.1%
2	3	< 0.1%
3	4	< 0.1%
4	4	< 0.1%
5	7	< 0.1%
6	5	< 0.1%
7	10	< 0.1%
8	8	< 0.1%
9	9	< 0.1%
10	7	< 0.1%

Value	Count	Frequency (%)
6082	1	< 0.1%
5358	1	< 0.1%
5189	1	< 0.1%
5050	1	< 0.1%
4930	1	< 0.1%
4855	1	< 0.1%
4769	1	< 0.1%
4616	1	< 0.1%
4490	1	< 0.1%
4372	1	< 0.1%

median_income
Real number (ℝ)

HIGH CORRELATION

Distinct	12929
Distinct (%)	62.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	3.870622563

Minimum	-0.4999
Maximum	15.0001
Zeros	0
Zeros (%)	0.0%
Negative	1
Negative (%)	< 0.1%
Memory size	161.4 KiB

Quantile statistics

Minimum	-0.4999
5-th percentile	1.60057
Q1	2.5634
median	3.5348
Q3	4.74325
95-th percentile	7.300305
Maximum	15.0001
Range	15.5
Interquartile range (IQR)	2.17985

Descriptive statistics

Standard deviation	1.89992041
Coefficient of variation (CV)	0.490856543
Kurtosis	4.951916681
Mean	3.870622563
Median Absolute Deviation (MAD)	1.0642
Skewness	1.646157328
Sum	79889.6497
Variance	3.609697566
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
3.125	49	0.2%
15.0001	49	0.2%
2.875	46	0.2%
2.625	44	0.2%
4.125	44	0.2%
3.875	41	0.2%
3	38	0.2%
3.375	38	0.2%
4	37	0.2%
3.625	37	0.2%
Other values (12919)	20217	98.0%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-0.4999	1	< 0.1%
0.4999	11	0.1%
0.536	10	< 0.1%
0.5495	1	< 0.1%
0.6433	1	< 0.1%
0.6775	1	< 0.1%
0.6825	1	< 0.1%
0.6831	1	< 0.1%
0.696	1	< 0.1%
0.6991	1	< 0.1%

Value	Count	Frequency (%)
15.0001	49	0.2%
15	2	< 0.1%
14.9009	1	< 0.1%
14.5833	1	< 0.1%
14.4219	1	< 0.1%
14.4113	1	< 0.1%
14.2959	1	< 0.1%
14.2867	1	< 0.1%
13.947	1	< 0.1%
13.8556	1	< 0.1%

median_house_value
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	3842
Distinct (%)	18.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	206855.8169

Minimum	14999
Maximum	500001
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	161.4 KiB

Quantile statistics

Minimum	14999
5-th percentile	66200
Q1	119600
median	179700
Q3	264725
95-th percentile	489810
Maximum	500001
Range	485002
Interquartile range (IQR)	145125

Descriptive statistics

Standard deviation	115395.6159
Coefficient of variation (CV)	0.55785531
Kurtosis	0.3278702429
Mean	206855.8169
Median Absolute Deviation (MAD)	68400
Skewness	0.9777632739
Sum	4269504061
Variance	1.331614816 × 10¹⁰
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
500001	965	4.7%
137500	122	0.6%
162500	117	0.6%
112500	103	0.5%
187500	93	0.5%
225000	92	0.4%
350000	79	0.4%
87500	78	0.4%
275000	65	0.3%
150000	64	0.3%
Other values (3832)	18862	91.4%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
14999	4	< 0.1%
17500	1	< 0.1%
22500	4	< 0.1%
25000	1	< 0.1%
26600	1	< 0.1%
26900	1	< 0.1%
27500	1	< 0.1%
28300	1	< 0.1%
30000	2	< 0.1%
32500	4	< 0.1%

Value	Count	Frequency (%)
500001	965	4.7%
500000	27	0.1%
499100	1	< 0.1%
499000	1	< 0.1%
498800	1	< 0.1%
498700	1	< 0.1%
498600	1	< 0.1%
498400	1	< 0.1%
497600	1	< 0.1%
497400	1	< 0.1%

ocean_proximity
Categorical

HIGH CORRELATION

Distinct	5
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	161.4 KiB

<1H OCEAN	9136
INLAND	6551
NEAR OCEAN	2658
NEAR BAY	2290
ISLAND	5

Length

Max length	10
Median length	9
Mean length	8.064922481
Min length	6

Characters and Unicode

Total characters	166460
Distinct characters	16
Distinct categories	4 ?
Distinct scripts	2 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	NEAR BAY
2nd row	NEAR BAY
3rd row	NEAR BAY
4th row	NEAR BAY
5th row	NEAR BAY

Common Values

Value	Count	Frequency (%)
<1H OCEAN	9136	44.3%
INLAND	6551	31.7%
NEAR OCEAN	2658	12.9%
NEAR BAY	2290	11.1%
ISLAND	5	< 0.1%

Length

Histogram of lengths of the category

Category Frequency Plot

Value	Count	Frequency (%)
ocean	11794	34.0%
1h	9136	26.3%
inland	6551	18.9%
near	4948	14.2%
bay	2290	6.6%
island	5	< 0.1%

Most occurring characters

Value	Count	Frequency (%)
N	29849	17.9%
A	25588	15.4%
E	16742	10.1%
	14084	8.5%
O	11794	7.1%
C	11794	7.1%
<	9136	5.5%
1	9136	5.5%
H	9136	5.5%
I	6556	3.9%
Other values (6)	22645	13.6%

Most occurring categories

Value	Count	Frequency (%)
Uppercase Letter	134104	80.6%
Space Separator	14084	8.5%
Math Symbol	9136	5.5%
Decimal Number	9136	5.5%

Most frequent character per category

Uppercase Letter

Value	Count	Frequency (%)
N	29849	22.3%
A	25588	19.1%
E	16742	12.5%
O	11794	8.8%
C	11794	8.8%
H	9136	6.8%
I	6556	4.9%
L	6556	4.9%
D	6556	4.9%
R	4948	3.7%
Other values (3)	4585	3.4%

Space Separator

Value	Count	Frequency (%)
	14084	100.0%

Math Symbol

Value	Count	Frequency (%)
<	9136	100.0%

Decimal Number

Value	Count	Frequency (%)
1	9136	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Latin	134104	80.6%
Common	32356	19.4%

Most frequent character per script

Latin

Value	Count	Frequency (%)
N	29849	22.3%
A	25588	19.1%
E	16742	12.5%
O	11794	8.8%
C	11794	8.8%
H	9136	6.8%
I	6556	4.9%
L	6556	4.9%
D	6556	4.9%
R	4948	3.7%
Other values (3)	4585	3.4%

Common

Value	Count	Frequency (%)
	14084	43.5%
<	9136	28.2%
1	9136	28.2%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	166460	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
N	29849	17.9%
A	25588	15.4%
E	16742	10.1%
	14084	8.5%
O	11794	7.1%
C	11794	7.1%
<	9136	5.5%
1	9136	5.5%
H	9136	5.5%
I	6556	3.9%
Other values (6)	22645	13.6%

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

The dendrogram allows you to more fully correlate variable completion, revealing trends deeper than the pairwise ones visible in the correlation heatmap.

First rows

	longitude	latitude	housing_median_age	total_rooms	total_bedrooms	population	households	median_income	median_house_value	ocean_proximity
0	-122.23	37.88	41	880	129.0	322	126	8.3252	452600	NEAR BAY
1	-122.22	37.86	21	7099	1106.0	2401	1138	8.3014	358500	NEAR BAY
2	-122.24	37.85	52	1467	190.0	496	177	7.2574	352100	NEAR BAY
3	-122.25	37.85	52	1274	235.0	558	219	5.6431	341300	NEAR BAY
4	-122.25	37.85	52	1627	280.0	565	259	3.8462	342200	NEAR BAY
5	-122.25	37.85	52	919	213.0	413	193	4.0368	269700	NEAR BAY
6	-122.25	37.84	52	2535	489.0	1094	514	3.6591	299200	NEAR BAY
7	-122.25	37.84	52	3104	687.0	1157	647	3.1200	241400	NEAR BAY
8	-122.26	37.84	42	2555	665.0	1206	595	2.0804	226700	NEAR BAY
9	-122.25	37.84	52	3549	707.0	1551	714	3.6912	261100	NEAR BAY

Last rows

	longitude	latitude	housing_median_age	total_rooms	total_bedrooms	population	households	median_income	median_house_value	ocean_proximity
20630	-121.32	39.29	11	2640	505.0	1257	445	3.5673	112000	INLAND
20631	-121.40	39.33	15	2655	493.0	1200	432	3.5179	107200	INLAND
20632	-121.45	39.26	15	2319	416.0	1047	385	3.1250	115600	INLAND
20633	-121.53	39.19	27	2080	412.0	1082	382	2.5495	98300	INLAND
20634	-121.56	39.27	28	2332	395.0	1041	344	3.7125	116800	INLAND
20635	-121.09	39.48	25	1665	374.0	845	330	1.5603	78100	INLAND
20636	-121.21	39.49	18	697	150.0	356	114	2.5568	77100	INLAND
20637	-121.22	39.43	17	2254	485.0	1007	433	1.7000	92300	INLAND
20638	-121.32	39.43	18	1860	409.0	741	349	1.8672	84700	INLAND
20639	-121.24	39.37	16	2785	616.0	1387	530	2.3886	89400	INLAND

Overview

Variables

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Decimal Number

Other Punctuation

Most occurring scripts

Most frequent character per script

Common

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Category Frequency Plot

Most occurring characters

Most occurring categories

Most frequent character per category

Uppercase Letter

Space Separator

Math Symbol

Decimal Number

Most occurring scripts

Most frequent character per script

Latin

Common

Most occurring blocks

Most frequent character per block

ASCII

Interactions

Correlations

Spearman's ρ

Pearson's r

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows